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1 . This declaration is to establish completion of the invention in this application in the 
United States, at a date prior to June 13, 1997, that is the effective date of the prior art: 

[X] publication 

[ ] patent 

that was cited by the 



[X] examiner. 



[ ] applicant. 



NOTE: "When any claim of an application or a patent under reexamination is rejected under 35 U.S.C. 
102(a) or (e) , or 35 U.S.C. 103 based on a U.S. patent to another or others which is prior art under 35 
U.S.C. 102(a) or (e) and which substantially shows or describes but does not claim the same patentable 
invention, as defined in 37 C.F.R. 1.601(n) , or on reference to a foreign patent or to a printed publication, 
the inventor of the subject matter of the rejected claim, the owner of the patent under reexamination, or the 
party qualified under §§§§ 1 .42, 1 .43 or 1 .47, may submit an appropriate oath or declaration to overcome 
the patent or publication. The oath or declaration must include facts showing a completion of the invention 
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in this country or in a NAFTA or WTO member country before the filing date of the application on which 
the U.S. patent issued, or before the date of the foreign patent, or before the date of the printed publication. 
When an appropriate oath or declaration is made, the patent or publication cited shall not bar the grant of a 
patent to the inventor or the confirmation of the patentability of the claims of the patent, unless the date of 
such patent or printed publication is more than one year prior to the date on which the inventor's or patent 
owner's application was filed in this country." 37 C.F.R. §§ 1.131(a)(1) . 

NOTE: 37 C.F.R. §§ 1.131 is not applicable to a rejection based on a U.S. patent that CLAIMS the rejected 
invention. 

2. The person making this declaration is (are): 
[ ] the inventor(s). 

[X] only some of the joint inventor(s), and 

[X] an authorized representative of the party in interest. The Assignment assigning the 
inventors' rights to the party-in-interest is recorded at Reel 9363, Frame 0766, recorded July 19, 
1998. 37CFR 3.73(b). The inventors no longer work for Samsung Information Systems America, 
an affiliated company of Samsung Electronics, Ltd., the party-in-interest. However, Jeffrey P. 
Aiello, Reg. No. 39,086, is the Patent Manager of the party-in-interest, and can best testify to the 
facts surrounding the procedures involved in the constructive reduction to practice of inventions, 
and the preparation of invention disclosures and patent applications by employees and 
contractors for the party-in-interest. 

FACTS AND DOCUMENTARY EVIDENCE 

3. 

To establish the date of completion of the invention of this application, the following attached 
document and/or models are submitted as evidence: 

(check all applicable items below) 

[X] A copy of United States Provisional Patent Application Serial No. 60/050,762 filed June 25, 
1997, including twenty nine (29) pages of text, images, flow charts and software coding 
specifically expressing how to implement the claimed invention 

[ ] sketches 
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[ ] blueprints 



[ ] photographs 

[ ] reproduction(s) of notebook entries 
[ ] model 

[X] supporting statement(s) by witness(es). Since this application has multiple inventors and a 
corporate assignee, each of the inventors and the assignee's representative are considered to be 
corroborative witnesses. 

(where verbal disclosures are the evidence relied upon) 

NOTE: "The showing of facts shall be such, in character and weight, as to establish reduction to practice prior to the 
effective date of the reference, or conception of the invention prior to the effective date of the reference coupled with 
due diligence from prior to said date to a subsequent reduction to practice or to the filing of the application." 37 
C.F.R. §§ 1.131(b). 

NOTE: While conception is the mental part of the inventive act, it must be capable of proof, such as by 
demonstrative evidence or by a complete disclosure to another. Conception is more than a vague idea of how to 
solve a problem. The requisite means themselves and their interaction must also be comprehended. See Mergenthaler 
v. Scudder 1897 CD. 724, 81 O.G. 1417." See also M.P.E.P. §§ 715.07 and §§2138.04 , 7th ed. 

From these documents and/or models, it can be seen that the invention in this application was 
made 

[]on . 

[X] at least before the date of June 10, 1997, which is a date earlier than the earliest possible 
asserted date of the reference. I have reviewed the attached Provisional Patent Application 
documents for USSN 60/050,762 filed June 25, 1997. These documents were signed by 
inventors Richard Humpleman and Kevin Harms on June 13, 1997, and by Robert Wolff and 
Michael Deacon on June 16, 1997. This is a 29 page document with extensive text, diagrams, 
flow charts, computer coding and other descriptions. This document was prepared in advance of 
June 10, 1997. The invention was conceived before June 10, 1997. The document evidences 
this because it was prepared after weeks, if not months of meetings and preparation of drafts for 
discussion prior to the final "Patent Disclosure" document being approved for Joint execution 
and filing. This document was then diligently circulated for execution and filing and then filed 
with the United States Patent and Trademark Office within two (2) weeks of execution. 



3 



NOTE: "If the dates of the exhibits have been removed or blocked off, the matter of dates can be taken care of in the 
body of the path or declaration." 

M.P.E.P. §§715.07, 7th ed. 

NOTE: "[ T ]he dates in the oath or declaration may be the actual dates, or, if the applicant or patent owner does not 
desire to disclose his or her actual dates, he or she may merely allege that the acts referred to occurred prior to a 
specified date." 

M.P.E.P. §§71 5.07, 7th ed. 

DILIGENCE 

NOTE: "Where there has not been reduction to practice prior to the date of the reference, the applicant or patent 
owner must also show diligence in the completion of his or her invention from a time just prior to the date of the 
reference continuously up to the date of the actual reduction to practice or up to the date of filing his or her 
application (filing constitutes a constructive reduction to practice, §§ 1.131)." M.P.E.P. §§ 715.07, 6th ed., rev. 3 
(emphasis added). 

NOTE: "A conception of an invention, though evidenced by disclosure, drawings, and even a model, is not a 
complete invention under the patent laws, and confers no rights on an inventor, and has no effect on a subsequently 
granted patent to another, UNLESS HE OR SHE FOLLOWS IT WITH REASONABLE DILIGENCE BY SOME 
OTHER ACT, such as an actual reduction to practice or filing an application for a patent. Automatic Weighing 
Mach. Co. v. Pneumatic Scale Corp., Limited 1909 C D. 498, 139 O.G. 991, M.P.E.P. §§ 715.07 , 7th ed. 

"Conception in the mental part of the inventive act, but it must be capable of proof, as by drawings, 
complete disclosure to another person, etc. In Mergenthaler v. Scudder, 1897 CD. 724, 81 O.G. 1417, it 
was established that conception is more than a mere vague idea of how to solve a problem; the means 
themselves and their interaction must be comprehended also." M.P.E.P. §§ 715.07 , 7th ed. 



NOTE: Only diligence before reduction to practice is a material consideration. The "lapse of time between the 
completion or reduction to practice of an invention and the filing of an application thereon" (Ex parte Merz 74 
U.S.P.Q. 296) is not relevant to an affidavit or declaration under 37 C.F.R. §§1.131 . M.P.E.P. §§ 715.07(a) , 7th 
ed. 

Attached is a statement establishing the diligence of the applicants, from the time of their 
conception, to a time just prior to the date of the reference, up to the: 

[ ] actual reduction to practice. 



[X] filing of this application. 

TIME OF PRESENTATION OF THE DECLARATION 
(complete (a), (b) or (c)) 

(a) [ ] This declaration is submitted prior to final rejection. 
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(b) [ ] This declaration is submitted with the first response after final rejection, and is for the 
purpose of overcoming a new ground of rejection or requirement made in the final rejection. 

(c) [X] This declaration is submitted after final rejection. A showing under 37 CRR. § § 1 - II 6(b) 
is submitted herewith- 

DECLARATION 

6. As a person signing below; 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made arc 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code , and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 

SICNATUHE(S) 

7, 

(complete A or B below) 
A. Inventor(s) 

+*ttt+*+ *+***+*+ 4 ******* ** + * *********************** ****************** ********* 



Full name of sole or first inventor: Richard Hunmlcman 

Inventor's signature: 

Efrte: , 



Residence; 343 Lower Vintners Circle. Fremont California 94539 



Cjtfapnship: British 



Post Office Adfress; Same as above 



****************************************t*****+***t+****++***++*******w***++++ 

nnmgn f sole or second inventor Q: Kevp MaAils A \ ; 

Inventor's signature: , / , f X? > l^w I -re^vv^ 

Pat:. |0/ UfO) 



Residence: S790 Bamawc ll Wav. Sari Jose. California 95 1 38 
Citi7.CTVihu?; T , T S 



Post Office Address: Same as above 
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(b) [ ] This declaration is submitted with the first response after final rejection, and is for the 
purpose of overcoming a new ground of rejection or requirement made in the final rejection. 

(c) [X] This declaration is submitted after final rejection. A showing under 37 CF.R, §§ L116(b) 
is submitted herewith. 

DECLARATION 

6. As a person signing below: 

1 hereby declare that all statement made herein of my own knowledge arc true and thai all 
statements made on information and belief arc believed to be true; and further that these 
siaiements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 1 8 of the United States 
Code p and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 

SIGNATURE(S) 

7. 

(completc A or B below) 
A. Inventor(s) 



run name oi sole or first invempj 
Inventor's signature: 


r; Kicn^o Jriumpicman 




Date: V 






Residence: 343 Lower Vintner^ 


Circle. Fremont. California 94539 


Citizenship: British 


Post Office Address: Same as above 



Full name of sole or second inventor: G. Kevin Harms , 

Inventor's si fmal^Te- 

Dalfii , 



Residence: 5790 Bar ps well Wav. San Jose. California 95 H8 
Citizenship; U.S. 



Post Office Address: Same aq pbove 
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95 West Piumeria Drive 
San Jo$e., California 95134 

Assignment recorded in PTO on July 19, 1998. 
Reel 9363 Frame 0766 

O: VKL S\SAM1 VDec-?TllU.D 1 4 
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(b) [ ] This declaration is submitted with the first response after final rejection, and is for the 
purpose of overcoming a new ground of rejection or requirement made in the final rejection. 

(c) [X] This declaration is submitted after final rejection. A showing under 37 C.F.R. §§ 1.1 16(b) 
is submitted herewith. 

DECLARATION 

6. As a person signing below: 

I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code , and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 

SIGNATURE(S) 

7. 

(complete A or B below) 
A. Inventor(s) 

*********************************************** 

Full name of sole or first inventor: Richard Humpleman 

Inventor's signature: 

Date: 

Residence: 343 Lower Vintners Circle, Fremont, California 94539 

Citizenship: British 

Post Office Address: Same as above 



****************************************************************************** 

Full name of sole or second inventor: G. Kevin Harms 

Inventors signature: 

Date: 

Residence: 5790 Barnswell Way, San Jose, California 95138 

Citizenship: U.S. 

Post Office Address: Same as above 
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B. Party-In-Interest 

Date: _ 

Jeffrey P. Aiello 
Reg. No. 39,086 
Patent Manager 

Samsung Information Systems America 
95 West Plumeria Drive 
San Jose, California 95134 

Assignment recorded in PTO on July 19, 1998. 
Reel 9363 Frame 0766 

G:\KLS\Saml\Dec-PRI0.914 
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B. Party-In-Interest 



Date: 

Jeffrey P. Aiello 
Reg. No. 39,086 
Patent Manager 

Samsung Information Systems America 
95 West Plumeria Drive 
San Jose, California 95 1 34 



Assignment recorded in PTO on July 19, 1998. 
Reel 9363 Frame 0766 

G:\KLS\SAMl\Dec-PRJ0.914 

CERTIFICATE OF MAILING BY "EXPRESS MAIL" l/Mlt)^ i 

I hereby certify that this paper of fee is being deposited with the United Stajis^pjsjal Eei^icdyW^ts- date: iV^ i y) * U ' 
envelope as "Express Mail Post Office to Addressee" Mailing Label Number -^3>5filfi RSODW addressed to: Box CPA, 
Commissioner for Patents and Trademarks, Washington, DX. 20231 ^ - — 



EVELYN MENJIVAR 



(Type or print name of person mailing paper) (Signature of person mfeilftig paper) 
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Docket Number 



2674-023PRO 
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LAST NAME 


FIRST NAME 
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[X] The Commissioner is hereby authorized to charge filing fees and credit Deposit Account 
Number 12-2237 
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[ ] Yes, the name of the U.S. Government agency and the Government contract number are: 



[] 
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Home Network, browser based, command and control: 
CLO-HTML/HTTP/IP/IEEE1394. 

Enabling DTV to render GUI's from and control CE devices with Com- 
mand Language Optional (CLO). 

Richard Humpleman; Multi-media Tech Centre, SAMSUNG, San Jose. 

tel: 408 474 3324, fax: 3353; email: richardh@sisa.samsung.com; file: siphot_patF.fm5 

Issue PROVISIONAL: JUNE 13th, 1997 



1.0 Background 

A couple of goals: 1) Stretch the standard use of HTML to its absolute limits prior to making 
extensions to that standard. 2) Put as few constraints on the target systems as possible. There are instances 
where dictating behavior is required, but outside of this, dictating behavior shall be limited as much as pos- 
sible. 

The heart of the "power of the web" is made up of two primary things: 1 ) Hyperlinking gives us the 
ability to get from anywhere to anywhere with one simple "reference" click. 2) Servers (and authors) are 
able to present new information, a new user interface, and many custom features to the masses who have 
web browsers as long as they stay inside the HTML spec boundries. This is a simple statement, but a very 
powerful notion. Historically, users have had to get "new" software on their computers in order to have 
these new "experiences". Never has there been a spec which is both simple and flexible which allows the 
world to choose one web browser piece of software to experience so much diverse material. Also, with the 
advent of server-side custom components, the user can be presented with simple HTML which activates 
complex server-side actions without the user knowing "what's behind the curtain". 

A key in understanding why these points are so important is to realize that this means each device 
is almost completelyresponsibile for its own actions and this makes it almost fully independent of all other 
devices. This is really the idea of 'encapsulation". 

As long as each device on the network has HTML files to describe their GUI and as long as they 
use HTTP protocol to transfer those files, then any "client" device that understands how to "web-browse" 
and render HTML will be able to use the device with the human- interface GUI. There are caveats to this 
such as automation and macro capabilities which do complicate things, but the power of the web does 
indeed lend itself very well to the home theater for such devices. The reason this works is that if a new 
"unknown" device is invented with a new function such as "instant video rental", all the new device has to 
do is present the HTML code which implements a button whose label is "Instant Video Rental" and when 
the user presses this button, the action taken is to "submit" or "post" information back to the originating 
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device. The device itself figures out which button was pressed and takes the appropriate action to cany out 
the action. The client system has no knowledge of how this happens and does not need to know (nor does it 
want to). 



The Home Theater is considered to be the model for digital equipment throughout the whole home, 
see figure 1 . This shows a collection of entertainment sources, audio, video and data networked together by 
a digital home network and one or more Digital TV 1 sink or display devices. The DTV provides the human 
interface for the whole system via GUI and the display. 

There cannot be only one network to satisfy all needs and so the diagram shows bridges to a Home 
Automation network and a 1 394 network that supports IEC 1 883 protocols. Another example (not shown) is 
an Ethernet Home Office Network. One of the reasons for choosing IP protocols is that it is a fully mature 
routeable inter-networking protocol that allows different networks in and outside the home to inter-operate. 

See Home-Network Protocol Proxy Appendix. 

The problem faced by the DTV designer is how to control the potential myriad of devices while 
keeping the unit simple, current and generic. 

The brute-force way is to build a DTV with knowledge of all the devices and GUI user interfaces 
for them. In addition one could develop a command set for the digital interface to enable remote control. 
One problem with this approach is that given the development rate of new devices it is impossible to keep 
the DTV GUI and Interface/Network command set from being complex and obsoleted. 

The approach chosen here is for the DTV to be a rendering Browser and bring in the Character of 
the device the user wishes to control, see figure 2. The device is represented by an 'html' (hyper text mark- 
up language) file kept in a accessible directory of the device. The 'html' file is an ascii text file with details 
of the device and infomation that enables a browser to present it graphically. 

In addition to 'bringing in 5 the html GUI to the DTV there is a return capability back to the device making 
the mechanism 2 way. The user can view the rendered html GUI and control the device by 'clicking' but- 
tons and form fill. 

The DTV browser accesses, using http protocol, the devices html file and renders it to create the 
devices GUI and present it to the user. The DTV can do this for any device but doesn't know what the 



l.See appendix for definition of DTV 
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Figure 1, HOME THEATER 



device type is -it is up to the human intelligence to read the GUI. This mechanism is used ordinarily for 
accessing information by computers communicating on the Internet and World- Wide- Web (www). In the 
case here, for device control, the html file represents the character of the device whereas in the www case 
the html file represents information. 

The device behaves like a server and the DTV behaves like a client. The client accesses the server 
initially to control it and later perhaps to receive it's video program stream. Typically the server has multi- 
ple html files in a file directory structure. In this case they may be partly device specific html and partly 
dynamically generated html from the media installed or online. The user is unaware of the physical source 
of the html GUI's eg some reside local to the DTV for it's own control to enable the switch between the 
overlay or window for the control browser and the overlay or window for the video program on the DTV. 

For simple cases the DTV is completely generic and there is no need for an interface command set. 
Moreover the system is compatible with the Internet protocols so may be controlled from a computer out- 
side the home running a browser just as well as the home DTV. 

As with the Internet case, the html file access use the http protocol that runs on the TCP transport 
protocol and IP network layer protocol. TCP provide a reliable delivery mechanism and IP the routeable 
addressing mechanism. The data transfers of audio and video program material are started by the html 
mechanism but run on the digital interface using hardware streams outside of the client server http/IP net- 
work based system. 
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Figure 2, HTML 2 Way Mechanism 



There are 2 cases where the html GUI command and control method isn't sufficient:- 
2) Machine and/or automatic control (see later). 

1) A client with no display capability (no GUI) -illustrated by the Audio Amplifier device. This is consid- 
ered to be a low order problem as there are other ways to get-around this problem. 

For these cases it is appropriate to have commands, buried in the html code, that are readable and 
writeable by software see later. 
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3.0 Architecture 



M Model 

For simplicity the architecture is described with 3 devices: DTV (Digital TV), DVCR (Digital 
Video Cassette Recorder) and DSS-NIU (DirecTV Satellite System-Network Interface Unit), see figure 3. 

The DTV is a Client as described above (client DTV unit also has to have server capability to 
enable access to the local hardware, see later). See appendix for more detail on the client model. 

The DSS-NIU is a mini-server (limited capacity) unit that can output a video program. The DSS- 
NIU is the video program tuner (satellite) separated into it's own unit called Network Interface Unit. Logi- 
cally this makes sense as both DVCR and DTV can now make use of the NIU capability. See appendix for 
more detail on the mini-server models. 

The DVCR is a mini-server unit as regards control. When recording (from DSS) the DVCR 
receives data and seems to be a client but this is a data transfer outside of the realm of client/server model. 



external 
port 



DSS-SERVER 
+ 

DHCP and 
NAME SERVER 



DVCR-SERVER 



DTV -CLIENT 
BROWSER 




Human 
I/F 



Figure 3, Architecture Devices 



3.2 Device and Media discovery 

The IP network protocol has to be automatically supported by the system. The unit least likely to 
be replicated is nominated to be the DHCP server (Dynamic Host Configuration Protocol) -in this system 
the DSS Server (though can be installed in any device of the home network). This is required in IPV4 (IP 
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network protocol version 4) to enable the network to work automatically and not require fixed IP addresses 
to be set up by the user. With future IPV6 this requirement will go away. 

See also appendix : Self Populating HTTP Server. 

As the DTV and DVCR power up they run DHCP client software that broadcasts on the network 
for the DHCP server. Once the DHCP server assigns IP addresses and names it updates the DSS-PC resi- 
dent, devices.html file with a hot link to each of the devices top-level html page.. This becomes the key file 
for user on the network to access devices, see figure 4. (ALL FILE NAMES GIVEN ARE ABRITRARY 
UNLESS OTHERWISE STATED). 
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BUTTONS 

TO 

LEFT 



ICON (MEDIA) 



Figure 4, Devices GUI -html 

When the DTV powers up the browser runs and displays the DTV top-level default html page 
which is the local user GUI file:///G/dtv/user.htmL This file is like the front panel control or remote con- 
trol of the TV. One of the buttons is "devices on home network". This accesses the URL http:// 
dhcp_server.xxx/devices.httnL (the actual access hot link would not specify the filename.html and would 
therfore access the machine_name/default.html). This must be a standard URL (hot link) for the home net- 
work the details of which (IP address and machine name) are completed by DHCP(cIient). For the case of 
the NIU, one physical unit which incorporates the DHCP server and DSS-NIU, there are 2 separate default 
files accessed by different IP addresses ie machine names. 

There is a case for obtaining a standard, top-level-domain 'dot extention' (machine.xxx) from 
InterNIC for the home network to clearly identify all local hot links and save unnecessary external internet 
access. 

The devices.html file accessed contains entries (buttons) for all available devices in the system. In 
this model it contains 3 entries (dtv, dvcr and dss). The user can now access all the devices, see 
devices.html GUI figure 4. 

A devices user.html GUI contains access to a structure of html files for additional purposes eg Set- 
^D Of device fsdiUStinSr bri^htPS c S l»*«»lc? *>tf*\' ec*\a-ri\nn r\rr\nr**rr> motorlol (T\T /-K™,-»^1V ~*^\s\„r* rt 
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macro; checking status; automatic control etc, see figure 5. 



LOGO (SAMSUNG 
DVCR) 



ICON (BRIGHTNESS, VOL- 
UME CONTROL ETC) 



ICON (FAVORITE PRO- 
GRAM CHANNELS) 



ICON (MAKE PROFILE OR 
MACRO) 



SELECT 

FROM 

BUTTONS 

TO 

LEFT 



ICON (BACK) 



Figure 5, DVCR example User GUI -html 

These human driven control actions take place entirely at the device using the GUI also from the 
device so an industry standardised command and control set isn't necessary. However, standard ascii com- 
mand set descriptors can be useful for scenarios involving machine driven (automatic) control. 

Note the use of LOGO and ICON graphic (GIF) flies. These allow the graphical company logo and 
icon device descriptor to be used. The GIF file names, sizes and compression type can be standardised to:- 
l°g° gif> s & e: 120x40 and icon.gif, size: 120x90, 

GIF compression is chosen because it is lossless, easy to render, and supports transparency and ani- 
mation (against is the limit of 256 colors). 



3.2.1 Device Page 

The Device page will list all of the devices on the Home Network with a link to each of the devices' 
top level HTML pages. The icon and logo image files will all be of a standard size so the list will can be 
arranged in an orderly manner. Preferably the device's logo would be displayed directly above the device's 
icon. The logo can act as a link to the device manufacturer's home page if so desired and the icon will be a 
link to the devce's top level HTML page. The images can be arranged in any manner desired by the NIU 
Server manufacturer, from as simple as a row and column configuration to a network topology diagram if 
possible. The NIU Server manufacturer might even allow the user to rearrange the images as they see fit 
and provide them with an additional text line below each device where the user can enter their own name or 
description for each device. For instance the user might be allowed to group devices by their location in the 
home with a name for each location (this kind of feature is entirely implementor driven and is not required). 
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3.2.2 



Logo Image Files 



A Logo image file is a file containing an image that represents the manufacturer of the device. It 
would typically contain an image with the name and logo of the manufacturer of the device. In order for the 
NIU Server to locate the file it must use the name icon.gif. The file must also be of a standard size, 1 20 x 40 
pixels, so the list of devices will have a neat, uniform look. Several variations of the logo file may reside on 
a device with a link to the desired file. The link can be updated over time or based on certain criteria of the 
manufacturer's choosing. The image may also be animated. 

3.2.3 Icon Image Files 

An Icon image file is a file containing an image that represents the type of device and potentially 
its state. It would typically contain an image with a picture of the device or a symbol that represents the 
type of device. In addition, a model number might be included at the bottom of the image to assist in iden- 
tification of the device on the Home Network. Several variations of the Icon file may reside on the device 
with each one representing a potential state. A link to one of the images would represent the current state of 
the device. To represent the various device states, the manufacturer has the choice of using a variety of 
symbols, colors, or even animation. (List some graphical examples below). The link may be updated over 
time or based on certain criteria of the manufacturer's choosing to indicate a change in state. Possible state 
values may be On, Off, Playing, Stopped, Recording, Rewinding, Forwarding, Searching, Media Inserted, 
or Media Absent. 

The purpose of the Icon image is to provide immediate device state information feedback to the 
user. In addition, since the Icon images are retrieved from all devices whenever the device list is displayed, 
there is an immediate indication of the accessibility of all devices on the network. In order for the NIU 
Server to locate the file it must use the name logo.gif. The image must also be of a standard size, 120 x 90 
pixels, so the list of devices will have a neat, uniform look. Note that it is up to the device to decide which 
of it's many ICON'S to substitute when asked for icon.gif. 

3.2.4 Location of files after discovery phase 

Figure 6 is a summary showing the location of files after the device discovery phase. Here each 
device now has a user.html file and the DHCP/Name Server has the devices. html In an actual implementa- 
tion these would both be named defaulLhtml . 
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external 
port 



DSS-N1U and DHCP/NAME 
SERVER 

dhcp_server/de vices.html 
dss/user.html 



DVCR MINI-SERVER 
dvcr/user.html 



DTV CLIENT BROWSER 
dtv/user.html 



Figure 6, Files Location after Device discovery and selection 



33 



Device set-up and control actions 



Use button 'clicks' and form fills to run programs and scripts on the device to make control 
actions. This is local and proprietary to the device -not performed remotely and therefore doesn't require 
any standardised 1394 command set. See figure 7 for location of the file and program components. 



external 
port 



DSS-NIU and DHCP-SERVER 

dhcp_server/dev ices, html 

dss/user.html 
s et -up_co ntro Lpro gram 



DVCR MINI-SERVER 

dvcr/user.html 
set-up_control_program 



DTV CLIENT BROWSER 

dtv/user.html 
set-u p_control_program 



Figure 7, Files/Programs Location after Device set-up and control program 

An example is given. The user may wish to change the brightness. On the User html GUI page the 
user can click on the brightness button. This may bring up another GUI with bright and dim buttons. If the 
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user clicks one of these the http server will cause a set brightness control program to run and make the con- 
trol hardware action. For action local to the DTV the DTV needs server capability -something to interpret 
the post actions from the browser. 

All home network DTV devices need server capability to be able to post actions to control their 
local hardware. This is important to understand. You can have a browser to pick up local html files and ren- 
der them to a GUI but this doesn't invoke the http server. Clicking on a button must involve an http access 
to the local machine name or IP address to invoke the local http server to respond which in turn invokes the 
local device (eg brightness) control program. 



3.4 Program selection 

Here additional html files are available to represent the programs audio and video material avail- 
able for the server device to source. They may be represented directly on the user GUI or down a level. 
They are represented as dss-channeLhtml and videofilchtml for the dss-niu and dvcr respectively. These 
html files are special as they are not at all static. The device updates the html file based on the dss EPG 
(Electronic Program Guide) and in the case of the dvcr the tape present in the machine. A program must 
exist as a go-between the source material content and the html file GUI available to the user. This is called 
the Dynamic Content_ControI_Program. See figure 8 for the location of files and programs. 



In order to reduce several, often used steps to a few easy steps, the use of macros as shortcuts is 
beneficial. Macro's are also used to store user Profiles or preferences. A macro is a sequence of commands 
that is saved in memory and is easily retrieved and executed at will. A macro is created by saving the com- 
mands that would normally be executed during a sequence of button pushes or actions by a user within the 
user interface. The macro is given a name so that it may be easily retrieved at a later time and executed. 
When the macro is executed it executes the sequence of actions in the macro just as if the user were select- 
ing buttons or performing actions from within the user interface. 

A macro is made and stored on the Server for which it is created. Because of the difficulties over- 
coming possible conflicts and deadlocks with other devices, a macro's scope is limited to the Server on 
which it is created. That is, it can only execute actions that pertain to the Server on which it was created. 
Therefore, when a macro is created it must be limited to commands that pertain only to the Server. Profiles/ 
macros across multiple machines are to be tackled at a later time. 

If the macro feature exists then the usenhtml contains a profile/macro button for generation on the 
server. Clicking this button starts the profile or macro recording by the macro generation program.The 
macro generation program monitors and saves all subsequent hotlinks accessed ('html issues') and 'return 
actions' for later replay. Ultimately the program results in the creation of another button the named profile 
or macro with hot-link: cgi bin/macro_for_user.htmL . 



Inventor's: ROBERT WOLFF, KEVIN HARMS, MIOHAEL DEACON, RICHARD HUMPLEMAN 



3.5 



Making a Profile or macro 



Inventor's Signature ((jO^:^^ 

Inventor's Signature. y^^/^^r../^I? 




Witness Name 



Signature 



Date. 



Witness Name 



Signature 



Date. 



page 11 of 29 

For more detail on macro generation see apendix. 



3.6 Checking status 

After a client (html fetch) and server (return-action) handshake the http spec server normally 
returns a status response code to indicate return-action good or not good (eg 200 returned indicates good 
and 400 or 300 returned indicates no-good). A bad response initiates a fetch of the status.html GUI this can 
include icon.giff\\cs that indicate graphically what the problem is. See figure 8 for the location of files and 
programs.Subsequently there can be a 'follow-up* action to access STATUS.HTML files generated and 
resident on the server with further information and suggested corrective action, see figure 9. 



DTV CLIENT BROWSER 

dtv/user.html 
set-up_control_program 
status.html 



DVCR MrNI-SERVER 




dvcr/user.html 




set-up_controI_program 




contcnt_control_program 




material/videofile.html 




status.html 





Figure 8, Files/Programs Location after Program selection and status 



external 
port 



DSS-NIU and DHCP-SERVER 

dhcp_server/devices.html 

dss/user.html 
set-up_control_program 
content_control_program 
materiaJ/dss-channe1s.htm1 
status.html 
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USER 

BUTTON 'CLICK' ON CHOICE 



STATUS 'CODE' TO USER 



> USER 



FOLLOW-UP 



ACCESS 
STATUS JNFO 
.HTML FILE 



SEND REQUEST 
STATUS_INFO.HTM 

RENDER 
STATUS.HTML 
GUI 



USER 'CLICK* STATUS INFO 



STATUS.HTML TO USER 



USER 



3.6.1 



Figure 9, Status code response and Follow-up status access 



Follow-Up status checking 



One of the inherent challenges associated with a digital network attached to consumer devices is 
the problem of multiple near-simultaneous access effecting the device. An example of this is that one NIU 
device could be setting the VCR's clock while someone at a DTV is telling the VCR to play a video while 
someone at a PC (at work possibly) is telling the VCR to "record channel 7 at 8pm tonight for 1 hour". 
Each of these activities has a status associated with the action. In the case of "atomic" operations, the status 
returned is either OK or NOT OK and that is all. In other operations such as rewinding a tape, the initial 
status may come back as "OK", but a status regarding how far along the rewind is or just if it has completed 
rewinding is needed via a status page. Another non-atomic example is a more complex one where the VCR 
has been set to record later tonight, but the user (now at work) wishes to change that setting or delete it alto- 
gether. This section describes an innovative way to handle these types of situations through multiple "status 
pages". 

When a client makes a connection to an http device, the client's IP address is given to the device so 
that the device knows where to send the requested information (HTML files usually). The idea here is to 
use that IP address as a unique identifier for making custom status files on the device for each client. So, in 
the above example, there would be three custom status files for "status.NIU_IPaddress.htm", "sta- 
tus.VCR_IPaddress.htm", and "status.DTV_IPaddress.htm". This gives each of those clients the ability to 
get status from the device which pertains ONLY to their client and no others. A generic "status.htm" file 
would contain hotlinks to each of the custom status pages as well as some other general status about the 



Inventor's: ROBERT WOLFF, KEVIN HA pi S, MICHAEL I 

Inventor's Signature...0^ \ 5Tr?^w-rvr^. . \^^rT>r>^-^ Date.... k. \)X L r Ll„ 

Inventor's Signature../^^ Date..^^/? 



DEACON, RICHARD HUMPLEMAN 



Witness Name Signature., 

Witness Name Signature.. 



Date.. 
Date.. 



page 13 of 29 



device (no tape inserted, 3 record_times currently set, low battery (if appropriate), power reset 20 minutes 
ago, etc.) So, to be clear though, remember that each device has an actual IP address such as 192.3.27.14 
and so making it unique. 



3.7 CLO-on-HTML/HTTP/IP/1394 



(CLO -Optional Command Language) 

An industry standard command set on top of html is useful for Automation and to a lesser extent 
GUI-less client devices eg Audio Amplifer. This allows software to review html flies for content and 
respond to make control actions. The command language is optional. Home devices and Digital TV work 
just fine with HTML/HTTP/IP/1394. 

A front-panel button press on the Audio Amplifier (involving an external device) is used for GUI 
control even though no GUI is displayed. Here the external device html is accessed and a parser reviews the 
html and select keywords. A control program selects the response return depending on the function 
required. 

Hot-links are standardised as commands eg source_seIect.html, increment__volume.html, 
bass_level.html, treble_level.html. The device can be now operated locally or remotely and can control 
other devices. 

3,7,1 Automatic Control (eg One touch record) 



Automatic control of the home network devices the layout of a set of necessary "control com- 
mands" for automation. This list, see table 1, is not an all-encompassing list of commands but an effort is 
made to make it as complete of a list as possible. This makes the irnplementors' job easier and clearer. 
Remember that many devices have no need to implement these commands. Only devices which have such 
functions or have a need to control other devices which have these functions have the need to implement 
specific functions from the list. 



Table 1: One Touch Record 



Name 


# Fields 


# Buttons 


Description 


TimeSet 


1 


1 (set) 


hhmmss (Local time) 


Record_Time 


4 


1 (set) 


ch#, time(hhmmss), len, mode? 


RecordTimeDelete 


2 


1 (delete) 


ch#, time(hhmmss) 
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Table 1: One Touch Record 



NcUDC 


ft r icjui> 


r>unons 


Description 


ivioae ociect 


1 


1 (set) 


For SP, LP 3 SLP modes. Should this be a 
drop-down instead? 


Stop 








FF 








Rew 








Play 








Record 









The home network model with separate NIU, VCR and DTV requires some automatic remote con- 
trol must take place across the network. Simple actions involving 2 devices eg dtv and a server are con- 
trolled by the simple html GUI mechanism described above. It is a simple action to select the dvcr GUI and 
play a tape installed to the dtv, however, setting the dvcr to record involves 3 devices the dss-niu, dvcr and 
the dtv where the control information entry takes place. One could set-up the dss-niu and then go and set-up 
the dvcr this would be 2 simple actions. However it is thought that the user would want an automatic one 
touch record system available. 



One touch record takesplace at the dss GUI where a selection is made for a future recording. Some- 
how the information must be transfered to the dvcr automatically. This is done by the dss server accessing 
the dvcr GUI automaticallyand filling in the record information and returning it back to the dvcr. 

This action involves an html GUI based command-set as a program in the dss-niu server must be 
able to scan the dvcr GUI for recognisable key words eg "RECORD TI ME" to enable it to fill in the time. 
This program needs to know it is accessing the dvcr GUI. Prior to this section the dtv accesses GUI's under 
human control and the machine had no knowldege of the device. 

The One Touch Record (OTR) program is triggered by the server observing a Vecord_program' set 
in the dss GUI. The OTR accesses the dvcr GUI transfers information from the dss GUI to the dvcr GUI 
and returns the dvcr form to set it to record, see figure 10. 
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DSS-NIU and DHCP-SERVER 

dhcp_server/devices.html 

dss/user.html 
set-up_control_program 
content_control_program 
material/dss-channels.html 
status.html 
otr program 



DVCR MINI-SERVER 

dvcr/user.html 
set-up _control_program 
content_con trol_program 
matcriaWidcofile.html 
status.html 
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DTV CLIENT BROWSER 

dtv/user.html 
set-up_controI_program 
status.html 



Figure 10, Files/Programs Location with OTR program 



3. 7. LI One Touch Record -How it works 

See appendix. 
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4.0 Conclusions 



The industry is currently looking a number of approaches to solving the home-network/Digital TV 
Interface problem. 



4.1 Approaches 

All approaches use IEC 1 883 for transport of isochronous data streams across IEEE 1 394 or special 
hardware control of 1394 isochronous streams for nbn-MPEG2 transport. There are 3 different approaches 
to controlling the program material streams as outlined below. 

4.1.1 Hardware-level command and control (1349AVC/IEC1883/IEEE1394) 

This approach without network layer addressing (eg IP) is limited in scope to local cluster control 
and data flow. The system is fixed to 1394 physical layer and a detailed hardware level command and con- 
trol specification must be standardised (and is under development) for all devices to use. 

Further functional and interconnect expansion can be done with proxy devices converting from 
network/application layer command and control to the AVC/1883 type command and control. 

The approach doesn't work well for a Home Theater DTV which is expected to control everything. 
A complex GUI would have to be accompanied by the detailed command set for every device making it 
difficult to design, expensive and quickly obselete. 

4.1.2 CAL/IP/IEEE1394 



This approach uses the command language CAL on the network layer IP so is much more general 
and flexible than 3.1.1 and not restricted just to 1394. 

The method seems not to solve the problem of DTV GUI availability for all current devices and 
obelescence regarding future devices and relies on remote control over the network layer IP. 

4.1.3 CLO-htmI/http/ip/1394 (SIPHOT approach) 

HTML/HTTP neatly solves the GUI problem by making the DTV a rendering browser and no 
interface command set is needed for human control of home network devices. An Optional Command Lan- 
guage (CLO) can be used for automatic machine control of devices (rather than human control). This takes 
the form of specific ascii commands on HTML/HTTP. 

One device is nominated to have knowledge of the home network devices to which all devices go 
initially for device or service selection. 
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5.0 Appendix 



5J DIGITAL TV (DTV) definition 

The Digital TV (DTV) is an display device (eg like a television CRT) and a box of electronics 
(STE -Set-top-electronics) containing the home network interface unit (NIU) eg IEEE 1394 digital inter- 
face, Digital video/audio decompression, D>A conversion, microprocessor controller to run control soft- 
ware, HTTP/IP protocol, browser software etc, see figure 1 1 . 



IEEE 1394 


BOX OF 
ELECTRONICS 


ANALOG 




DISPLAY 
DEVICE 
(eg CRT) 




(STE) 


r 



Figure 11. DTV definition 
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5,2 ONE TOUCH RECORD (how it works) 

The way this automation works is through the use of HTML forms and the ability for systems to 
automatically "simulate" user-input of values into form fields and the clicking of a "set" button. An exam- 
ple would be helpful here. Let us take the example of setting a VCR to record a program from 8pm-9pm on 
Friday night June 6, 1997 on channel 7. In the "normal" HTML GUI, the VCR would produce HTML code 
which would allow the user to put this information into the form and then click on a button to set this pro- 
gram. A very simple non-graphical version of this GUI would look something like table 2. 



Table 2: VCR AUTOMATIC RECORDING SET-UP 





Please fill out forms and 
click on set 






CHANNEL 




DATE (mm/dd/yy) 




START TIME (hhmm) 




START AM/PM 






SET 



This basic form gives all the information necessary to set the VCR to record Friday night. HTTP 
protocol makes our automated process simple. If each "command" has a unique name, then we implement 
our automation by doing the following: 



Call the "action" /command/<commandname> (in this case /command/recordtime) and each field 
gets a standard name. Then when the automated device wishes to automatically setup a record time as 
above, it simply prepares the following "POST" command. 



POST /command/record time HTTP/1 . 1 
Content- Type: text/plain 
Content-Length: 47 

channel=07&date=06/06/97&starttime=0800&ampm=pm 

Generically stated, commands/methods are handled like this: 

POST /command/standard_commandname HTTP/1.1 
Content-Type: text/plain 
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Content-Length: calculated_argumentlength_below 

object l=value&object2=value[&objectn=value....] 

This simple bit of text is pushed back to the device which is to be set for recording. Then the target 
device responds with some status information regarding whether or not the requested record time was OK/ 
valid or not. The status issues are dealt with in a separate section of the document. One of the interesting 
points to note here is that the "automatOR" device does not have to request any documents in order to pro- 
gram the VCR for recording tomorrow night. As long as the automatOR device knows which command 
they wish to invoke, they simply gather the information necessary to carry out the command (object=value 
arguments) and then issue the approriate http POST command and wait for an http status response back. 

Indeed this does mean that we carry the initial burden of defining a fair number of commands for 
general use in order to cover a large percentage of the needs of the device community. This, however, is an 
acceptable burden as it is not too difficult to define these commands. 
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5.3 



DVCR Mini-Server Model 



The DVCR-PC functions as a mini-http server, see Figures 12. 
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Figure 12. DVCR MINI-SERVER MODEL 



5.3.1 DVCR-PC Features 

PC andOS with HTTP server capability and TCP/IP PROTOCOL Stack. 

HTML user GUI for Samsung DVCR (user.html ) and video content on the tape/media (videofile.html) 
Accessible MPEG Transport file(s) 

MPEG Tpt D(DMA) output to 1394 isochronous from remote http command using CGI-BIN program 
Update of content html (videofile.html) by program that can be started on command or by media insertion. 
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5.4 



DSS/DVB and DHCP-server Mini-server Model 



The DSS-NIU functions as a mini-http server, see Figures 13. The DHCP-server is also shown here 
though this may reside with any unit on the network. 



file access for DSS-NIU eg 
/user.html 




HAV control incl. control 
of MPEG2 Transport 



file access for content eg 
/videpg.html 



to/from 

DTV 

(client) 



PROTOCOL 
STACK 
7. - HTTP 

4. -TCP 
3. -IP 
1/2.- 1394 



1394 interface 



data access for DSS eg 
/video.mpeg 



DMA Data 
to 1394 
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Figure 13.. DSS-NIU MINI-SERVER MODEL 
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5A1 



DSS/DVB-NIU, DHCP-Server and Internet Features 



PC andOS with HTTP server capability and TCP/IP PROTOCOL Stack. 
CGI-BIN program to access the hardware 

CGI-BIN program to access the off-air EPG hardware and system 
Program to convert accessed DSS/DVB EPG data to html GUI form. 

Program to take program specific EPG data and convert to html form (eg DISNEY channels only GUI) 
File access for videpg.html (video EPG data GUI) 
(EPG=Electronic Program Guide). 

CGI-BIN program to access the MPEG-2 transport hardware 

DSS/MPEG (transport) off-air video program data output to 1394 isochronous from http command (to 
MPEG2-Transport-over-1394 spec eg IEC1883) possibly using DMA. 

Executables to access the DSS/DVB h/w from http command 

File access for the dss/dvb-user.html -the GUI shipped with the device for device control. 



5.4.1.1 For DHCP server function 

DHCP (IP address discovery/assignment) Program 

HTML GUI converter/generator of devices present (devices.html) 

File access for devices.html 



5.4.1.2 For Internet Access Function 

Internet access Proxy, 
Internet Firewall 
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5,5 DTV Client Model 

The DTV client browser human interface is shown in Figure 14. The browser renders GUI (graph- 
ical user interfaces) but does not source them. In the demo SIPHOT the source is the Mini-Server (NIU or 
DVCR). The Browser has 'hot-links' that result in a new GUI and executables can be trigerred to run on the 
server or client. 

Hot links beginning http:// access the mini-server. If the link is also to cgi-bin (or *.asp) then any 
executable referenced in the html script will be executed on the server. Hot links beginning Tile' are 
accessed on the client DTV only. Hot links to DTV(self) that are required to perform hardware action, must 
address the DTV server by having a bonafidi HTTP hot link to the DTV (self). This of course requires that 
the DTV also has HTTP server capability. 
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BROWSER GRAPHIC 



DSS/DVB-PLAY 
http://dssplay.html 



DVCR-PLAY 
http://cgi-bin/rundvcr.html/ 



DTV 

file:///c|/cgi-bin/rundtv-hw.exe/ 
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7. - HTTP 
6. 
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Figure 14. CLIENT MODEL 



to/from 
DSS NIU 



1394 interface 
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5J> Self Populating HTTP Server 

5.6.1 Problem 



From a consumer-viewpoint, all network devices should "simply work" by plugging the device 
into a network 'outlet 1 and turning the device on. It should be immediately useable etc. The desire from the 
user would be to have all network devices have logical names and be able to be used in a singular user 
interface which is both consistent and easy to use. In addition, Power-users also would like to be able to 
control, configure, and monitor the network in a consistent and easy-to-use fashion. 

5.6.2 Solution 



An HTTP server is at its roots simply a collection of files in a heirarchical storage 'tree 1 . Users gain 
access to various parts of the 'tree 1 by browsing through hyperlinks or by being pointed to a specific loca- 
tion in the tree from som external source. At the "root" of the tree, there is a general "welcome" page 
(home-page user interface). The branches of the tree are usually configured, created, and managed by one 
or more "webmasters". 

In this solution, the custom-programmed webserver self-modifies its tree by creating branches for 
each device on the network as they are discovered and registered with the central naming authority (DHCP 
server). This requires cooperation between the self-modifying http server and the DHCP server. After dis- 
covering a new device, the HTTP server queries the new device for its capabilities and managability specif- 
ics. It then builds a new sub-tree-branch for this new device. Then, the "welcome" page is automatically 
updated with this new device information "hotlink". 

At this point, the user is able to "browse" through the network of devices from one central HTTP 
server and is able to control, configure, and monitor them since this central HTTP server has intimate 
knowledge of each device on the network. 

One of the more innovative pieces to this self-populating tree is that it has the ability to begin cate- 
gorizing and indexing available (and unavailable) media for the home. By this, we mean that the server 
knows at any given time (via polling for status), which devices have which media inserted. For instance, in 
an advanced (expensive) home setup with multiple AV clusters, Dad puts a DVD movie in one of the DVD 
players in the bedroom and later that evening, he does not have to remember which DVD player it was 
inserted into nor if it was DVD, VCR, DVCR, etc. Using the self-populating tree, Dad finds the title he was 
interested in and it would play regardless of its physical location. This includes the case where Dad's son 
physically moves the DVD movie to another location for some unknown reason. Such a system requires 
devices to have insertion and removal notification to the DHCP server in order for it to keep track of which 
devices have what media. This indexing and categorization technique is also available to older media as 
well given a little "help". For instance, audio CDROM disks each have a unique ID number which can be 
associated in a database to the actual artist and title via a database. Even track titles, lyrics, etc. can be 
obtained by databases. It is our belief that with such a system (and standard) in place that the record label 
companies will provide the consumer with web-access to such data. This makes the 200+ CD juke-boxes 
much more compelling to purchase as the user gets to choose their songs to play via the television set or 
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1 

\ 
\ 



PC. 



Welcome to your Home Network 



Buttons for Known devices are listed below: 
TV 

D-VCR 
DVD 

Den-Computer 
Kids-Computer 
DSS 

Network Configuration/Testing 
Network Monitoring 
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A macro is stored on the device for which it is created. Because of the difficulties overcoming pos- 
sible conflicts and deadlocks with other devices, a macro's scope must be limited to the device on which it 
is created. That is, it can only execute local commands. Therefore, during macro creation, only local com- 
mands can be enabled. This is a task that is left up to the developer of the macro generation routine. 

5.8.3 Is Simultaneous Setup and Control Possible? 

Various devices from different manufacturers may exist on a Home Network simultaneously. In order to 
facilitate convenient setup and control of several devices in tandem, macros may be used. When the macro 
is executed it executes.the sequence of actions in the macro just as if the user were selecting buttons or per- 
forming actions from within the user interface. A macro is not limited to storing actions from one user 
interface, but can be used to store actions from a sequence of menus and various user interfaces. 

In a Home Network environment the situation can be made even more complicated by a prolifera- 
tion of devices that require simultaneous control and by devices that are under the control of several other 
devices or users (OOPS, Deadlock). (Need to resolve possible conflicts and deadlocks. Macro #1 is going 
to record from the DSS and gets to a point where it needs access to the VCR, but the VCR is being used by 
macro #2 that is recording Hawaii Five-0 from the DVD. So while macro #1 waits, macro #2 stops record- 
ing from DVD and tries to record from DSS, but macro # 1 has control of the DSS.) * 

For example, setting up a VCR to record from a DSS. One could leave the DSS on all the time and 
set the VCR to record at a particular time, but that's an awkward solution. The DSS not only consumes 
more power and experiences unneeded wear when left on for extended periods, but may be inadvertently 
switched to a different channel or innocently turned off before the VCR is able to record the desired pro- 
gram. 

* - Any failure to acquire resources teminates the macro. 
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